Controlling data processing

ABSTRACT

Data processing apparatus comprises a control device having one or more user-operable controls; a pressure detector to detect user pressure applied while operating the one or more user-operable controls; and a processor to initiate a data processing action in response to a detection of user pressure of at least a threshold amount greater than that required to operate the one or more user controls.

BACKGROUND Field of the Disclosure

This disclosure relates to controlling data processing.

Description of the Prior Art

Some data processing activities may be controlled by a detection of atrackable device, for example.

An example arrangement involves a games machine, in which a gamesmachine is connected to a device such as a head mountable display (HMD)or a display monitor and operates in response to one or morehand-holdable controllers such as a Sony® Move® Controller, a Sony® DualShock® controller or the like.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described embodiments, together with further advantages,will be best understood by reference to the following detaileddescription taken in conjunction with the accompanying drawings.

SUMMARY

Various aspects and features of the present disclosure are defined inthe appended claims and within the text of the accompanying descriptionand include a data processing apparatus, a method, a control device anda computer program.

Example embodiments of the present disclosure provide data processingapparatus comprising:

a control device having one or more user-operable controls;

a pressure detector to detect user pressure applied while operating theone or more user-operable controls; and

a processor to initiate a data processing action in response to adetection of user pressure of at least a threshold amount greater thanthat required to operate the one or more user controls.

Example embodiments of the present disclosure also provide a controldevice comprising:

one or more user-operable controls;

a pressure detector to detect user pressure applied while operating theone or more user-operable controls; and

a processor to initiate a data processing action in response to adetection of user pressure of at least a threshold amount greater thanthat required to operate the one or more user controls.

Example embodiments of the present disclosure also provide a methodcomprising:

a pressure detector to detect user pressure applied while operating oneor more user-operable controls of a control device; and

initiating a data processing action in response to a detection of userpressure of at least a threshold amount greater than that required tooperate the one or more user controls.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 schematically illustrates use of a games machine or console;

FIGS. 2 and 3 schematically illustrate example controller devices;

FIG. 4 schematically illustrates a controller device;

FIGS. 5a-5c and 6a-6b schematically illustrate control buttons;

FIGS. 7 to 10 are schematic flowcharts illustrating respective methods;

FIGS. 11 and 12 are schematic flowcharts illustrating respective methodsrelating to machine learning techniques;

FIG. 13 schematically illustrates a data processing device or apparatus;and

FIG. 14 is a schematic flowchart illustrating a method.

DESCRIPTION OF THE EMBODIMENTS

Referring to the drawings, FIG. 1 schematically illustrates a systemcomprising a display 100 such as a head-mountable display (HMD) and/or amonitor connected to a games machine 110 such as a Sony® PlayStation®games console 300 as an example of a base device.

FIG. 1 also illustrates a hand-held controller 120 which may be, forexample, a Sony® Move® controller or a Sony® DualShock® controller or aSony® Sixaxis® controller which communicates wirelessly with the gamesconsole 110 to control (or to contribute to the control of) operationsrelating to a currently executed program at the games console.

A camera 130 is associated with the console or games machine 110 tocapture images of the user and/or the controller 120.

Examples of the controller 120 are shown schematically in FIGS. 2 and 3.FIG. 2 schematically illustrates, as an example of a hand-heldcontroller 120, a PlayStation 3 game controller 200, known as theSIXAXIS® controller, which comprises two joysticks 250, 260, two sets offour press-buttons 230, 240, and two pairs of pressure sensitive triggerbuttons, 211, 212, 221, 222. In addition the central portion of thecontroller contains three ‘system’ buttons 270 that typically accessfunctions of the operating system or current application and a set 290of LED indicators.

In addition, the controller 200 contains a MEMs(micro-electro-mechanical systems) motion detection mechanism (notshown), comprising accelerometers (not shown) that detect the extent oftranslational movement along three axes, and gyroscopes (also not shown)that detect the extent of rotational movement about these axes.

Another example controller 300 (FIG. 3) schematically represents aso-called DualShock® controller which shares many similar functions anduser interfaces with the controller 200 but also provides a touch pad310 and a haptic feedback arrangement (not shown) to provide vibrationalor other feedback to the user holding the controller 300.

Another example type of controller (not shown) is a so-called Sony®Move® controller which comprises a handle portion and an illuminated endportion. The handle portion 100 can carry one or more control buttonsand/or joysticks and houses an inertial measurement unit (IMU).

Referring to FIG. 4, some aspects of the structure of one of thecontrollers 200, 300 are shown. In FIG. 4, haptic feedback arrangementsand motion sensing arrangements are not shown, because these particularfeatures are not relevant to the techniques to be discussed below.

In FIG. 4, an example control button 400, touch pad 410, joystick 420and LED indicator 340 are shown, all of which are connected to controlcircuitry 440 which in turn communicates with a wireless interface 450representing an example of wireless communications circuitry tocommunicate with the processor (440 or 110).

A power supply 460 supplies power to the various circuit components.

In use, the user operates one or more buttons, touchpads, joysticks,from which operations the control circuitry 440 generates controlsignals to be sent by the wireless interface to the games machine orapparatus 110. The games machine 110 performs a data processing action,such as progressing a particular aspect of computer gameplay, inresponse to the information received from the controller 120. As well asgenerating information for display by the display arrangement 100, thegames machine can also communicate back to the controller, via thewireless interface 150 and the control circuitry 450, information to becommunicated to the user, such as lighting up one or more LEDs 430 oroperating the haptic feedback arrangement if present (not shown).

Processing functions associated with parsing and interpreting the useroperations of the controls 400, 410, 420 may be carried out by thecontrol circuitry 440 and/or by a processor at the games machine 110.The question of where the processing is carried out is a routine designchoice for the system designer. Carrying out more processing of thecontrol information at the controller itself (for example using thecontrol circuitry 440) can help to reduce wireless data usage by thewireless interface 450 because raw data does not need to be sent.However, the computer processors associated with the games machine 110are generally much more powerful than processors which can be housed ina hand-held controller, and also the power supply arrangements availableto the games machine 110 are generally significantly greater than thoseprovided by the power supply 460 (for example a rechargeable battery).So this would suggest carrying out more processing activity at the gamesmachine 110. But as discussed above, the choice of where to do thisprocessing is a design choice for the system designer.

Each of the types of controls found in FIG. 4, namely buttons 400, touchpad 410, joystick 420 may be made pressure-sensitive. Example techniqueswill be discussed in relation to buttons but similar techniques can beapplied to any types of controls.

FIG. 5a schematically represents a control button for use in thecontroller 110, comprising a finger-pressable formation 500 (the“button” part which is touched by the user) which is configured todeform a conductive sheet 510 from a rest position as shown in bold linein FIG. 5a to a deformed position 520 shown in broken line. Deformingthe conductive sheet 510 closes a circuit between an electrode 530 andthe conductive sheet 510, so that this can be detected as a button pressby suitable electronics.

Referring to FIG. 5b , a pressure detector 540 can be positioned betweenthe formation 500 and the sheet 510, or in FIG. 5c , a pressure detector550 can be mounted with respects to the electrode 530 so as to detectdownwards (as drawn) pressure on the electrode 530.

The pressure detector 540, 550 may be, for example, a piezo-electricdetector, a detector formed of a block or sheet of metal-loaded orcarbon-loaded conductive foam having a resistance which varies accordingto compressive forces applied to the foam, or the like. An electricalsignal indicative of the amount of user pressure can be provided to thecontroller 440 by the detector 540, 550.

FIG. 6a schematically illustrates another type of control button inwhich a metal-loaded or carbon-loaded flexible plastics cap 600 may bepressed down onto a pair of electrodes 610, 620 by user pressure on abutton portion 630 and in doing so it completes a circuit between theelectrodes 610, 620. The position 600 is a rest position of thedeformable cap and a position 5640 pressed down upon the electrode 610,620 is a deformed position when the button is pressed. Here, userpressure can in fact be detected by a change in the resistance betweenthe electrodes 610, 620. The resistance goes from infinite (when thebutton is not pressed) to a finite value as the button is pressed so asto apply the conductive cap 600 onto the electrode 610, 620, but maythen continue to drop as the button is pressed even harder. So, thebasic switch arrangement of FIG. 6a can provide a pressure indication.In FIG. 6b , a discreet pressure detector 650 is provided so as todetect the application of force by the user pressing the button portion630.

Note that in other examples, instead of (or in addition to) the sensor650, a sensor portion 660 can be included as part of the buttonstructure. A generally equivalent example is applicable in FIG. 5b ,where the sensor effectively forms a lower (as drawn) part of the buttonstructure.

Equivalent arrangements can be applied to the other types of usercontrol discussed above. In the context of a control device comprisingone or more user-operable controls (such as buttons, joysticks, touchpad controls), this provides various examples of a pressure detector540, 550, 650, 660 to detect user pressure applied while operating theone or more user-operable controls; and a processor 440 (though some orall of this functionality may be provided by 110) to initiate a dataprocessing action in response to a detection of user pressure of atleast a threshold amount greater than that required to operate the oneor more user controls.

So, each of these arrangements can provide a pressure-related signal tothe control circuitry 440. In these examples, one or more of theuser-operable controls comprise a respective pressure sensor.

FIGS. 7-10 schematically illustrate possible outcomes relating to thedetection of the pressure-related signal and in particular with respectto the detection of an over-pressure situation where the user is deemedto be pressing “too hard” on one or more of the control buttons. Thedefinition of “too hard” is one which can be predetermined or can bederived by the controller and/or the games machine, for example, apredetermined upper limit of acceptable user pressure (consistent withlong-term usability of the controller 110) may be established, but as asecondary threshold, a level of user pressure which is (say) 50% greaterthan the user's normal level of pressure needed to operate the controlsmay be applied as a threshold. Therefore, in examples, the test appliedto detect an over-pressure situation can be, for example, a detection ofuser pressure of at least a threshold amount (as a pressure differenceor a proportion) greater than that required to operate the one or moreuser controls.

Example responses (data processing actions, for example by the gamesmachine 110) will now be described with reference to FIG. 7-10, forexample involving one or more actions selected from the list consistingof:

notifying the user;

notifying one or more other users;

initiating a testing procedure of at least the control device; and

storing data indicative of a current data processing state of theapparatus at the time of the detection.

Each of the schematic flowcharts of FIG. 7-10 starts with a step 700,800, 900, 1000 of detecting such an over-pressure situation. Variousoptions are available to handle such a detection. For example, in FIG.7, at a step 710, the system assumes that there is a potential faultwhich is causing the controller to be temporarily relativelyunresponsive, and so a self-test process is initiated by the gamesmachine and/or the control circuitry 440.

In FIG. 8, the user is sent a notification at a step 810 to inform theuser (visually, audibly by haptic feedback or the like) the user is atrisk of damaging the controller 110.

In FIG. 9, an assumption may be made that the reason the user ispressing the controller harder than usual is because of user excitement,thereby indicating a particularly thrilling or engaging section ofgameplay, so in response the system (for example the games machine 100)records at a step 910 an instance of the gameplay state at that time.

In FIG. 10, at a step 1010, other user(s) may be notified of theinstance of over-pressure, again using the over-pressure as a proxy forthe recognition of a particularly engaging piece of gameplay, so thatother users or even commentators may wish to know about it.

The processor 440 and/or the processing function (in this context) ofthe games machine 110 may be performed by, for example, a trainedmachine-learning processor. FIG. 11 schematically illustrates a trainingprocess for such a processor and FIG. 12 schematically illustrates aninference process in operational use of such a processor.

Referring to FIG. 11, the training process is performed with respect toso-called ground truth training data 1100. This can include ground truthinput data such as pressure data from user application of thecontrol(s), game state data, microphone (audio) data providing anindication of the user's state of agitation or the like. The groundtruth output data is an indication (obtained by observing or questioningthe user or by having created artificially situations in which thecontrols became relatively unresponsive by virtue of a software or areal or simulated hardware fault) as to the cause of the over-pressureand therefore the appropriate outcome of those shown in FIGS. 7-10. Thismay comprise a significant number (perhaps, many thousands) of dataacquisitions, some indicating game excitement, some indicating faultycontrols or software and the like.

During the training phase, an outcome (FIGS. 7-10) is inferred usingmachine learning parameters such as machine learning weights. At a step1120, an error function between the outcomes associated with the groundtruth training data 1100 and the inferred outcomes at the step 1110 isdetected, and at a step 1130, modifications to the parameters such asmachine learning weights are generated and applied for the nextiteration of the steps 1110, 1120, 1130. Each iteration can be carriedout using different instances of the ground truth training data 1100,for example.

In an inference phase of the trained machine-learning processor (FIG.12), over-pressure data (and optionally other data such as game statedata, audio (microphone) data or the like is detected at a step 1200,and then, at a step 1210, an outcome (FIGS. 7-10) is inferred using thetrained machine learning parameters generated as described above.

FIG. 13 schematically illustrates a data processing apparatus or devicesuch as the games console 300. The apparatus 1500 comprises one or moreprocessing elements such as a central processing unit 1510 (CPU), anon-volatile memory 1520 such as a read only memory (ROM), a flashmemory, a magnetic or optical disk or the like, a random access memory(RAM) 1530, a user interface 1540, a network interface 1550 to provideconnectivity with different apparatus in the system, and (at least forthe control apparatuses) a camera interface 1560. All of these elementsare interconnected by a bus structure 1570. In operation, program codeis stored in the non-volatile memory 1520 and transferred to the randomaccess memory 1530 for execution i.e. CPU 1510 to perform thefunctionality discussed above.

FIG. 14 is a schematic flowchart illustrating a method comprising:

detecting (at a step 1400) user pressure applied while operating one ormore user-operable controls of a control device; and

initiating (at a step 1410) a data processing action in response to adetection of user pressure of at least a threshold amount greater thanthat required to operate the one or more user controls.

It will be appreciated that in example embodiments the techniquesdiscussed above, including the method of FIG. 14, can be implemented bycomputer software operating on a general purpose computing system suchas one or more games machines which may have the form of the apparatusof FIG. 13. In these examples, computer software, which when executed bya computer, causes the computer to carry out any of the methodsdiscussed above is considered as an embodiment of the presentdisclosure. Similarly, embodiments of the disclosure are provided by anon-transitory, machine-readable storage medium (such as the medium1520) which stores such computer software.

It will also be apparent that numerous modifications and variations ofthe present disclosure are possible in light of the above teachings. Itis therefore to be understood that within the scope of the appendedclaims, the disclosure may be practised otherwise than as specificallydescribed herein.

1. Data processing apparatus comprising: a control device having one ormore user-operable controls; a pressure detector to detect user pressureapplied while operating the one or more user-operable controls; and aprocessor to initiate a data processing action in response to adetection of user pressure of at least a threshold amount greater thanthat required to operate the one or more user controls.
 2. Apparatusaccording to claim 1, in which one or more of the user-operable controlscomprise a respective pressure sensor.
 3. Apparatus according to claim1, in which the data processing action comprises one or more actionsselected from the list consisting of: notifying the user; notifying oneor more other users; initiating a testing procedure of at least thecontrol device; and storing data indicative of a current data processingstate of the apparatus at the time of the detection.
 4. Apparatusaccording to claim 1, in which the data processing apparatus is acomputer games apparatus.
 5. Apparatus according to claim 1, in which:the control device comprises wireless communications circuitry tocommunicate with the processor.
 6. Apparatus according to claim 1, inwhich the one or more user operable controls comprise one or moreselected from the list consisting of: buttons; joysticks; and touch padcontrols.
 7. Apparatus according to claim 1, in which the processorcomprises a trained machine-learning processor.
 8. A control devicecomprising: one or more user-operable controls; a pressure detector todetect user pressure applied while operating the one or moreuser-operable controls; and a processor to initiate a data processingaction in response to a detection of user pressure of at least athreshold amount greater than that required to operate the one or moreuser controls.
 9. A method comprising: detecting user pressure appliedwhile operating one or more user-operable controls of a control device;and initiating a data processing action in response to a detection ofuser pressure of at least a threshold amount greater than that requiredto operate the one or more user controls.
 10. A non-transitorymachine-readable storage medium which stores computer software which,when executed by a computer, causes the computer to perform a methodcomprising: detecting user pressure applied while operating one or moreuser-operable controls of a control device; and initiating a dataprocessing action in response to a detection of user pressure of atleast a threshold amount greater than that required to operate the oneor more user controls.